Controllable lyrics-to-melody generation

Lyrics-to-melody generation is an interesting and challenging topic in AI music research field. Due to the difficulty of learning correlations between lyrics melody, previous methods suffer from low quality lack controllability. Controllability generative models enables human interaction with generate desired contents, which especially important tasks towards human-centered that can facilitate musicians creative activities. To address these issues, we propose a controllable lyrics-to-melody network, ConL2M, able realistic melodies user-desired musical style. Our work contains three main novelties: (1) model dependencies attributes cross multiple sequences, inter-branch memory fusion (Memofu) proposed enable information flow multi-branch stacked LSTM architecture; (2) reference style embedding (RSE) improve as well control generated melodies; (3) sequence-level statistical loss (SeqLoss) help learn features given lyrics. Verified by evaluation metrics for controllability, initial study shows better feasibility interacting users styles when